A tracing system is an important tool used during development and contributes to the efficient testing and verification of hardware and software projects. In general, the tracing system records the behavior of the hardware or software implementation during a period of operation. The chronological event trace produced by the tracing system illustrates the control logic and data flow as an application or test is performed. In many tracing system, the event trace information captured while the tracing systems is operating is stored in a separate trace file for subsequent inspection and analysis.
Together the tracing systems and resulting event trace can be used in a variety of ways. For example, the tracing system can be used to identify “bugs” in hardware or software causing errors or system checks. The event trace provides clues for determining the cause of the errors and helps verify when error has been resolved.
In another application, the tracing system and event trace can be used to refine and improve an existing hardware and/or software design. The tracing system monitors an existing hardware and/or software implementation and generates a trace that is studied and analyzed to better understand the control logic and data flow. This helps confirm that the hardware and software systems operate as expected and gives insight to refining the design and improving performance.
Existing tracings systems on the market typically rely on a combination of hardware and software subsystems. Hardware generally has physical connections built into the system for reading, writing, or monitoring for tracing events. For example, a logic analyzer receives digital information from the bus of a device and converts the signals received into information that can be readily analyzed. The software portion of these tracing systems track a sequence of events generated while a software system is running. For example, the trace information generated from the software system shows instructions being processed by the system, the content of registers, cache information, and variable information used in the software system being traced or debugged.
Unfortunately, these existing tracing systems are inadequate in multiprocessor or distributed processing environments. Chronological information used to sequence information is not readily available as several processors are capturing events using different clocks or synchronization mechanisms. Even if the clocks on the various systems are initially synchronized, over time the clocks drift and the trace events may start to appear out of order or incorrect. Accordingly, lack of synchronization makes it difficult to combine trace events from multiple processors or sources in these distributed systems. Consequently, it is difficult to analyze and trouble-shoot operation of these distributed systems.
Without adequate tracing systems, it is difficult to create high speed and reliable distributed processing systems. For example, distributed storage controllers interoperating with fiber channel switches (FCS) and storage devices to store and retrieve data from storage area networks (SAN) cannot be rapidly developed. Distributed network controllers and processing systems are also difficult to develop and support.